Automation Model Flow Control
The exact order and number of tasks executed by an automation model can be controlled with the use of the Preconditions, Events, and Connector elements.
Sequence
A sequence in an automation model is a series of connected model elements joined together. As an automation model looks like a flowchart, a straight sequence of tasks is sometimes referred to as a path, pathway, or flow.
- A sequence or path is the entire set of objects and connectors from one component to another
- There can be only one direct path from one ActionTask to another.
- The direction of the connecting arrows defines the order in which tasks and sub-Runbooks are executed. Such tasks and sub-Runbooks are executed in serial.
Split Path
A split occurs where a single Task or Precondition has outgoing connections to more than one model object.
- The automation model splits from one path into multiple paths, leading to many possible sequences.
- At least one of the pathways must be followed until the End node, otherwise the model will abort or time out.
- The tasks that are performed after a split are executed in parallel.
- The reverse of a split, where a single model object has multiple incoming connections, is a merge.
Merge
Merging occurs when two or more pathways converge in the automation model and connect to a single object. Merging can occur before a Task, End, or Precondition. All of these types of objects have a Merge property that lets you control when the automation model continues to the object. Each independent path usually executes and arrives at the receiving object at different times.
There are three merge types:
- All
- Any
- Targets
The display value for the Merge type is configured through the automation.merge.policy.default
system property. Valid values for it are ALL
and ANY
. Targets
is not a valid value. Set the default to the value that you use most often.
The merge type that you select in the object's Properties Panel always takes precedence over the value in the system property.
Merge All
Merge All means that the object at the merge will wait for all merging paths to complete before executing.
Merge All is the default option.
Merge Any
Merge Any means that the object at the merge will execute as soon as one of the merging paths completes.
Merge Targets
Merge Targets is a special merge case used when a sequence of tasks is executing with multiple target instances. A task set to Merge Targets will wait for each target to complete before proceeding.
For more information about targets, see Targets.
Troubleshooting Merges
An incorrect merge type can cause undesired results. When troubleshooting Runbook execution flow, check if merges are set to the intended type. If two paths are merging into a Merge All task, but control flow logic dictates that only one of these paths will execute, the Runbook will hang and eventually time out.
In the following example, the automation execution traverses only one of the mutually exclusive paths. The default merge type (All) does not produce the desired result because the task waits for both paths to complete. Since only one path completes, the process eventually times out and aborts.
Merge types will also affect how variable data is inherited from incoming tasks to the merging task. Different paths in an automation have independent FLOW variables. When the paths merge:
- The merging task gets its FLOW values from either the first (only) task to finish (if Any)
- or the last task to finish (after waiting for all incoming tasks to complete on All).
In the following example, the automation splits into two paths, as shown in the diagram:
- Each path inherits the same FLOW variables but taking different times to complete
- The left path (yellow, with variable values of A) takes longer to complete and arrives last to the merging Task 4.
Execution Dependency
Execution dependencies are used to control automation flow based on ActionTask completion. By default, completion of a task is required. If a task fails to complete execution for any reason, the process will not continue down that path. If there is no alternative path, the process eventually times out and aborts. In some cases, it may not be required that the preceding task complete.
- Execution: this dependency specifies the criteria.
- Dependency: Click the relevant Connector object and then select from the Dependency options.
The Execution dependency options are:
- Required (default) - The path will only be followed if the preceding task completes (represented by a solid Connector line).
- Not Required - The path will always be followed (represented by a dashed Connector line).
Condition Dependency
Condition dependencies are used to control automation flow based on ActionTask assessment results.
- The Assessor script of an ActionTask sets a condition variable (the RESULT. Condition) with a value based on the determined success of the task.
- Connector paths are assigned condition dependencies (set with the table options) and are followed only if the preceding ActionTask result condition matches the dependency setting.
The Condition dependency options are:
- Not required (default) - The path will be followed regardless of the task condition (represented by a standard grey arrow).
- Good (Success) - The path will be followed if the task condition is Good (represented by a green arrow).
- Bad (Failed) - The path will be followed if the task condition is Bad (represented by a red arrow).
It is possible for a task to have multiple outgoing pathways with a particular dependency. In the following example, the green path is followed if the assessed task is successful, while the process proceeds down both the middle and right paths (both red) if the task fails.
Critical Path
You can mark a path in your automation as critical for reporting purposes. Having a path marked as critical does not have any implications on the automation's workflow and logic but is required for the generation of the Critical Path Report.
See Reports to learn how to generate a Critical Path Report.
Do the following to mark the critical path:
- On the path that you want to mark, place a Task and set it up to run the
RunbookPathProfiler#Reports
ActionTask. - Ensure that the task's
PATH_MARKERS
input variable is set toSUCCESS
either during execution or as a custom value.
The Critical Path Report assumes that the automation has taken another path if it doesn’t find the SUCCESS
keyword.
Precondition Element
A precondition is a decision point, or a fork, in the automation flow. The evaluation of the condition that you provide determines which pathway is executed. Using a precondition is beneficial when a single ActionTask condition is not enough to determine the correct path.
Another reason to use a Precondition instead of logic inside an ActionTask is to keep the ActionTask as reusable as possible. You can decide whether the returned value is good or bad as part of the ActionTask logic but this makes it less flexible as the good/bad threshold might vary from automation to automation. A better option is to use a Precondition to compare a FLOWS
map value to a given threshold to decide which path the automation takes.
Incoming and Outgoing Connectors
You can have as many incoming and outgoing connectors to a Precondition as you need.
The incoming connection type does not affect how the Precondition works.
Outgoing connectors act the same way as on the Task element:
- If the Precondition expression evaluates to true, the Automation takes the green (Good) path as long as it is unnamed.
You can have multiple Good connectors leading out of the Precondition. - If the Precondition expression evaluates to false, the Automation takes the red (Bad) path as long as it is unnamed.
You can have multiple Bad connectors leading out of the Precondition. - Grey (None) paths are always followed as long as they are unnamed.
You can have multiple None connectors leading out of the Precondition. - For named connections, including named Good or Bad connections, the Precondition expression must evaluate to a string matching the connection name for the path to be taken.
Restrictions on Preconditions
A Precondition must lead to one of the following element types:
- Task
- Automation
- End
- Precondition
Preconditions do not act instead of ActionTasks, and are not considered intervening task objects. This specificity is important when connecting ActionTasks with multiple paths. Remember that only one direct path can run from a given ActionTask to another. A path with a Precondition on it is still viewed as a direct path as shown on the next diagram. Both path 1 and 2 are considered direct paths from the initial task to the Task if False
object.
You can run into this situation anywhere two pathways of any type merge, so be careful with your merge conditions. The merging task might not execute and the Worksheet results may show two End
tasks. The simplest remedy is to add a noop
ActionTask on one of the paths. A noop
task performs no operation (it constitutes a wait
task with wait time of zero) but constitutes a task.
Merging at a Precondition
Merging occurs when two or more pathways converge into the Precondition object. The merge dependency controls when the automation model continues and the precondition executes. As with the other elements supporting merging, you have three options to choose from for the Merge property: ALL (set by default), Any, and Targets.
The default display value of Merge is managed through the System Properties page using the automation.merge.policy.default
system property. The merge option that you select in the Merge drop-down in Precondition Properties always takes precedence over the default system property value.
Creating a Precondition
To create a precondition, take these steps:
- In the Automation editor, drag and drop the Precondition element from the Designer Panel on to the canvas.
- With the Precondition element selected on the canvas, in the Precondition Properties panel on the right, fill in the fields as follows:
- Display Name—Enter a descriptive name for the precondition that will appear on the canvas.
- Merge—Select the merge type: ALL, Any, Targets. See Merge for option descriptions.note
Similar to the ActionTask's Merge option, the precondition's default-display Merge value is configured through the
automation.merge.policy.default
system property (Main Menu > System Administration > System Properties). You might want to change the default to save time for selecting the option manually for each precondition during development (for example, set it toANY
when the majority of the preconditions that you create have a Marge type ofANY
). - Switch to Advanced Mode/Wizard Mode—Enables you to choose how you enter a precondition expression:
- Advanced Mode—Enter an expression using Groovy code.
- Wizard Mode—Build an expression using visual controls. note
Partial expressions that you’ve entered in either mode are preserved if you switch modes.
- Expression—Only in Advanced Mode Enter a valid Groovy expression.
- Precondition Wizard—Only in Wizard Mode Use visual controls to build an expression step by step.
Creating an Expression in Wizard Mode
In Wizard Mode, you use visual controls to select the variable to act upon as well as the comparison, boolean, or string operation to apply to the variable. In the background, the wizard generates Groovy code that you can view if you switch to Advanced Mode.
You can include multiple facets in a precondition expression that you chain using logical operators.
All expressions created using Wizard Mode evaluate to boolean. Use Advanced Mode if you need to evaluate to a named connection's name.
Your automation needs to meet at least one of the following conditions for you to be able to build preconditions in Wizard Mode. If it doesn’t, you will not have any variables to choose from in the Field field.
- It features at least one Task element that runs an ActionTask with at least one Input or Output parameter mapped to a
PARAM
,FLOW
, orWSDATA
variable. - If has at least one Automation Parameter.
There are no restrictions on the parameter type.
To create a precondition expression in Wizard Mode, take these steps:
- Fill in the form fields as follows:
- Precondition Wizard—The expression appears in this box as you build it.
Use the minus controls at the end of each facet to remove facets that you don’t need. - Field—Select a variable to act upon. The drop-down list shows all Automation Parameters and mapped ActionTask parameters. note
You can reference more variables in Advanced Mode.
- Operator—Select the operator to apply to the variable. The drop-down list contains a selection of boolean, comparison, and string operators.
- Value—Enter the value to compare to. Does not apply to boolean operators.
- Logical operator—Select the logical operator to use to chain the current facet to the previous.
This field is ignored on the first facet that you enter because there isn’t a previous facet to chain to.
- Precondition Wizard—The expression appears in this box as you build it.
- Click Generate to create a single expression facet using the values you entered.
- Repeat these steps to create as many facets as you like.
Creating an Expression in Advanced Mode
In Advanced Mode, you use Groovy to write the precondition expression. You can write any Groovy code as long as it evaluates to:
- Boolean:
- A true result means taking the Good (green) path out of the precondition. The path needs to be unnamed.
- A false result means taking the Bad (red) path out of the precondition. The path needs to be unnamed.
- String matching an outgoing path name, in which case the path (outgoing connection) matching the expression result will be taken.
Only Groovy expressions that meet the Wizard Mode criteria can be converted and edited in Wizard Mode. You receive an error when you try to switch to Wizard Mode from a more elaborate Groovy expression.
Example
In the following example, we want to check the free disk space on a remote machine.
First, we develop an ActionTask that does the following in its Assess script:
- Determines whether the task has successfully logged in and obtained the required information.
- Writes the result (amount of free disk space) into a
FLOWS
variable.
In the interest of creating a reusable ActionTask, we do not want the task to decide whether the returned value is good or bad because that threshold might vary. Instead, we use a Precondition to compare the FLOWS
value to a given threshold to decide which path the automation takes. For this example, we want to check if the free disk space is at least 80 GB.
FLOWS.diskSpace > 80
The second part of the expression makes a null check, which is always a good idea. After the expression evaluation, if the value is indeed 80 GB or more, the Automation takes the Good path, where you could end the Automation or print an affirmative comment using the comment#resolve
ActionTask. A value under 80 means the Automation takes the Bad path where you could create an Task to remediate the issue on the remote machine.
Targets
Targets are used to execute an ActionTask or series of tasks multiple times concurrently (on multiple "targets", i.e. execution instances). For example, a single ActionTask can use targets to execute a "ping" for a set of IP addresses concurrently during Runbook execution. The QUEUE_NAME option is a one-time setting for a particular task, while TARGETS affects all subsequent tasks in a Runbook sequence until the Merge Targets dependency.
If it is not necessary to execute an ActionTask a variable number of times, with dynamic parameters, consider these alternatives:
- Branching or splitting in an automation model - You need to set the required number of executions at the time the model is created.
- Looping – it allows for a variable number of sequential executions.
Branching and merging within targets - If the automation is branched after targets are created, be sure to Merge All or Merge Any back to a single path before Merge Targets is reached.
Conceptual Diagrams of Targets
Automation models with targets can be represented visually as having multiple splits at the node that creates targets:
- Where simple branching splits the model horizontally across the screen or page.
- Splitting into targets layers each target or execution instance vertically.
In the following figure, the basic Runbook includes no targets and executes the sequence only once, flowing through either Path 1 or Path 2.
The same Runbook can include targets to run the process multiple times simultaneously. In the next figure, the first ActionTask (TASK A) sets the targets in the Preprocessor. Each target is evaluated separately at TASK B, the node with the condition dependency, and each target might take a different path.
In the next figure, the second ActionTask (TASK B) sets the targets, but each target is still assessed individually. TASK A executes before TASK B in the sequence, and thus runs only once.
Keeping track of targets in an automation model can be difficult. Consider implementing color-coding which can be set with the Format option.
Mechanics of Targets
Targets are set by the Preprocessor before the task is executed. Groovy code can be used in the Preprocessor to calculate how many targets to set and their parameters. For example, the Preprocessor logic can decide how many task executions are needed based on the INPUTS. Before setting targets, the TARGETS variable must be refreshed using:
TARGETS.clear()
Set targets by adding parameters to the TARGETS variable. Label each desired target with any unique string. With the NAME as the only required parameter, add each target with the call:
TARGETS. Add("NAME");
Optionally, each target can be assigned a queue name and a map of parameters:
TARGETS. Add("NAME", [:]);
TARGETS. Add("NAME", "QUEUE", [:]);
The QUEUE parameter is a named queue on which the task should execute. This allows the queue to be dynamically determined by the Preprocessor, instead of being hard-coded in the task definition. For example:
TARGETS. Add("NAME", "MY_CUSTOM_QUEUE", ["KEY_1":"VALUE_1"]);
If no queue name is passed, the tasks will execute on the default RSREMOTE queue. The parameter map will be available as PARAMS in the target execution. This allows different parameters to be passed to each execution.
Loops
A process may require multiple iterations of the same task or series of tasks to detect, diagnose, or resolve a problem. Execution of a task can be repeated using loops. Using loops is also preferred when an external occurrence needs to be waiting for too long.
By default, the automation model does not allow loops. Looping can be enabled for an individual automation by setting the PROCESS_LOOP
property to "true" in the Start element. With Inputs selected, go to Edit to bring up Edit Parameters.
- If the model contains loops but
PROCESS_LOOP
remains set to "false", the automation will not execute. Checkrscontrol.log
for errors. - When using loops in sub-Automations, the parent Start task must have
PROCESS_LOOP
set to "true".
Loops vs. Targets
Both loops and targets are used to invoke multiple executions of the same tasks or sequences. Loops are preferred when the automation:
- Requires "serial" execution (limit the number of processing threads)
- Uses shared resources
- Is implementing events
Targets are preferred when the situation:
- Calls for "parallel" execution (multi-threading)
- Uses no shared resources (such as connections)
- Uses no events
Loop Guidelines
Creating loops in Resolve Actions Pro should follow the same guidelines as in any programming language. Using them is efficient but only when done carefully. The reason is that loops may create problems and it is sometimes better to consider using their alternatives.
Before using loops:
- Create an exit strategy:
- Provide an exit condition out of a loop to avoid creating non-terminating loops;
- Set a loop counter as a backstop strategy. This means to define a maximum number of iterations that will stop a loop. Use a GLOBAL, FLOW, or WSDATA variable, which persist in an execution path.
- The Failed/Bad non-terminating (repeating loop) condition should lead directly to the loop counter.
- Analyzing for an exit path, if the normal exit path returns to the loop, then a problem can occur.
- Consider boundary conditions:
- Avoid tight loops as they use an inordinate amount of processing power. If a loop runs too fast, add a wait to slow it down.
- Use cascading logic (have multiples of the same Task element) as opposed to loops when only a few iterations are required. If necessary, use a container to hide cascaded logic to simplify the view of the automation model.
- If iterating a list, account for 0 items, 1 item, and 1+ items;
- Keep track of data handling such that the list size decreases with each iteration;
- The task to which the loop returns must use Merge: ANY.
In the next example, the "parse" Task only enters the loop if data is present. The Precondition checks if the loop should continue and exits if not.
Queues
To predictably control which RSRemote instance executes an ActionTask, configure QUEUE_NAME
. This can be done while editing the ActionTask. When an ActionTask is executed, the request is sent to a queue. One of the RSRemotes listening to that queue picks up the request and begins processing the task.
Queues can also be set dynamically using the Preprocessor and the Targets property. The specified queue is used for all executions of the ActionTask unless overridden by Targets.
Using Named Queues
Each RSRemote is configured to listen to one or more queues. When installed, all RSRemotes are configured to listen to the "RSREMOTE" queue. When a gateway is configured, it automatically adds its own queue name to the RSRemote configuration. For example, an RSRemote running the Netcool Gateway will be configured to listen to the "RSREMOTE" queue and the "NETCOOL" queue.
It is also possible to configure user-defined queues. This allows you to set up RSRemotes for specific tasks. For example, if the RSRemote is installed on Windows, use a queue named "WINDOWS".
Configuring Named Queues
Configuration of queues for an RSRemote is done using the "blueprint.properties" file and the configuration command. The set queues can be seen in the config.xml file for the RSRemote (also accessible in GUI as part of List Registration in the System Administration menu). Each RSRemote in the cluster can have different queue names configured. To add a custom queue, go to the blueprint file and locate the line:
rsremote.esb.queue.name.1=RSREMOTE
This defines the default "RSREMOTE" queue. Other queues can be added using the following format:
rsremote.esb.queue.name.2=WINDOWS
rsremote.esb.queue.name.3=NETCOOL
In this example, the RSRemote can execute any task sent to the "RSREMOTE", "WINDOWS", and "NETCOOL" queues.
Multiple RSRemotes may connect to the same named queue, but a task sent to that queue will be executed by only one of them.
Examples of Named Queues
Named queues are necessary in certain situations.
- Windows ActionTasks must be performed by an RSRemote running on a Windows platform. Because Actions Pro is deployed on UNIX, the default RSRemote cannot be used to execute Windows ActionTasks.
- Segmented Networks exist where an RSRemote bridges a centralized network and a remote network. Devices on the remote network cannot be accessed from the centralized Actions Pro server and can only be accessed from the RSRemote server that is connected to the remote network(s).
- Customer Networks/Overlapping Networks/NAT Environments - for multiple overlapping or NAT network environments, RSRemote can be deployed on the remote network or site and provide access to the remote network devices.
- Access Control Systems - for systems, applications, and networks that can only be accessed by a specific access host, for example MAINFRAME, MIDRANGE, etc.
Removing the Default Queue
RSRemotes that must respond immediately to specialized task requests should be configured to stop listening to the default queue. Removing the default queue frees the RSRemote from processing "general" tasks. For example, a JUMPHOST RSRemote is reserved for tasks that require the jump host.
Queues vs. Targets
Other than using the QUEUE_NAME
option, queues can be set dynamically using the ActionTask Preprocessor and the Targets variable. The specified queue in QUEUE_NAME
is used for all executions of the ActionTask unless overridden by Targets. Using the QUEUE_NAME
option (and thus setting a default queue) is preferable to using Targets to set a queue name for a single task execution.
The ActionTask QUEUE_NAME
option and the TARGETS.add
variable behave similarly in an automation model. To run a task, they both require at least one RSRemote config.xml
to have the specified queue name configured. They differ in that the QUEUE_NAME
option is a "one-time" setting for just that particular task, while TARGETS affects all subsequent tasks in a Runbook. Also, TARGETS
. Add can be called more than once in a Preprocessor to execute the task multiple times concurrently.